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ABSTRACT 


Leaders  in  both  the  military  and  civilian  sectors  make 
a  series  of  interrelated  decisions  in  real  time  to  achieve 
goals.  These  decisions  involve  the  allocation  of  resources, 
such  as  ships  and  aircraft  to  influence  the  situation  facing 
the  decision  maker.  NEWFIRE  is  a  computer-based  simulation 
of  a  forest  fire  fighting  task  that  allows  the  experimenter 
to  control  both  the  goals  and  the  environment  in  which  the 
decisions  are  made  and  thereby  explore  the  effects  these 
variables  have  on  the  decision  maker. 

The  objective  of  this  thesis  was  to  use  the  NEWFIRE 
microworld  to  determine  the  effects  that  multiple  goals  and 
system  complexity  have  on  decisions.  Specifically,  subjects 
were  given  one,  two  or  three  goals,  and  confronted  with 
three  scenarios  of  varying  complexity.  The  results  show 
that  subjects  given  only  one  objective  outperformed  those 
given  two  or  three  objectives.  The  results  also  show  that 
the  performance  of  subjects  on  the  most  complex  scenario  was 
worse  than  on  the  less  complex  scenarios. 
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I .  IHTRODUCTION 


A.  RESEARCH  QUESTION 

Individuals  make  decisions  for  the  purpose  of  achieving 
some  goal  (Brehmer,  1992;  Hogarth,  1981).  Whether  that  goal 
is  as  mundane  as  preparing  a  meal  or  as  critical  as  winning 
a  naval  engagement,  a  series  of  interrelated  decisions  is 
required.  These  decisions  must  be  made  in  real  time  in  an 
environment  that  changes  of  its  own  accord  as  well  as  in 
response  to  the  decisions  made  (Brehmer,  1992;  Sengupta  and 
Abdel-Hamid,  1992).  Typically,  laboratory  studies  have 
tended  to  freune  decision  making  in  static  contexts  (Hogarth, 
1981) . 

Brehmer  (1992)  suggests  the  use  of  microworlds  as  an 
effective  means  to  investigate  the  continuous  nature  of 
decision  making  in  complex  dynamic  environments.  A 
microworld  is  an  interactive  computer-based  simulation.  An 
example  of  a  microworld  based  on  a  forest  fire  fighting  task 
is  NEWFIRE  (Loevborg  and  Brehmer,  1991).  Brehmer  (1992,  p. 
238)  argues  that  the  results  of  experiments  conducted  thus 
far  with  microworlds  "...may  express  some  generally  valid 
characteristics  of  people  trying  to  control  complex  dynamic 
systems . " 

Microworlds  have  been  used  to  address  issues  of 
cognitive  heuristics  and  feedback  strategies  (Kleinmuntz, 
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1985)/  complexity  of  the  environment  (Mackinnon  and  Wearing, 

1980) ,  decision  strategies  (Kleinmuntz  and  Kleinmuntz, 

1981) ,  learning  the  time  constants  of  a  task  (Brehmer  et 
al.,  1992a)  and  incorporating  cost  information  in  decision 
making  (Brehmer  et  al.,  1992b).  Brehmer  (1992)  notes  that 
goal  conflicts  seldom  appeared  in  these  experiments. 
Therefore,  while  the  results  are  generally  indicative  of  the 
subjects'  mental  models,  an  important  aspect  of  decision 
making  still  needs  to  be  incorporated  into  the  research 
(Brehmer,  1992). 

This  thesis  employs  NEWFIRE  to  address  the  following 
research  question:  How  does  the  number  of  goals  affect  the 
quality  of  decision  making  in  complex  dynamic  environments 
under  different  levels  of  task  complexity? 

B.  HYPOTHESES 

Two  primary  hypotheses  guide  the  research  question.  The 
first  hypothesis  concerns  the  relationship  between  the 
number  of  goals  a  decision  maker  is  attempting  to  meet  and 
the  quality  of  decisions  he/she  makes.  Because  some  goals 
may  conflict  with  each  other  and  because  the  process  of 
tracking  progress  toward  more  than  one  goal  is  cognitively 
more  challenging,  subjects  given  more  goals  should  make 
decisions  of  lesser  quality  than  those  given  fewer  goals. 
Decisions  of  poorer  quality  should  generally  lead  to  worse 
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performance.  Thus  the  first  hypothesis  is:  Subjects 
receiving  one  goal  will  perform  better  than  subjects 
receiving  two  goals,  who  will  perform  better  than  subjects 
receiving  three  goals. 

The  second  hypothesis  concerns  the  relationship  between 
the  complexity  of  the  task  and  the  quality  of  decisions  made 
in  the  completion  of  that  task.  Complexity  in  NEWFIRE 
arises  from  the  number  of  fires,  the  wind  direction  and 
velocity,  and  the  efficacies  of  the  fire  fighting  units. 
These  characteristics  have  been  incorporated  in  three 
experimental  scenarios,  labelled  as  easy,  moderate  and 
difficult.  These  scenarios  are  described  in  detail  in 
Chapter  III  and  Appendix  A.  Lesser  complexity  means  fewer 
elements  and  fewer  interactions  that  must  be  considered  by 
the  decision  maker.  It  is  therefore  reasonable  to  expect 
that  subjects  would  perform  better  on  the  cognitively 
easier,  less  complex  tasks.  The  second  hypothesis  is: 
Subjects  will  perform  better  in  the  easy  scenario  than  in 
the  moderate  scenario,  and  better  in  the  moderate  scenario 
than  in  the  difficult  scenario. 
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II.  THEORETICAL  PREMISE 

A.  DTMAMIC  DECISIOH  EMVIROMMERTS 

The  private,  public  and  government  organizations  in 
which  individuals  must  make  decisions  are  complex,  and  that 
complexity  is  growing  (Mackinnon  and  Wearing,  1980). 
Mackinnon  and  Wearing  (1980)  note  that  this  complexity 
primarily  stems  from  three  sources: 

1.  the  number  of  elements, 

2.  the  degree  of  interaction  eunong  those  elements,  and 

3.  the  degree  to  which  uncertainty  influences  the  system. 
The  decisions  individuals  must  make  are  also  dynamic  in 
nature  (Sengupta  and  Abdel-Hamid,  1992).  Decision  makers 
must  make  decisions  in  response  to  the  demands  of  the 
environment.  An  individual  cannot  make  decisions  only  when 
he/she  feels  ready  to  do  so  (Brehmer,  1992). 

Medical  decision  making  is  a  good  example  (Kleinmuntz 
and  Kleinmuntz,  1981).  The  goal  of  the  physician  is  to 
diagnose  and  cure  the  patient.  The  physician  must  examine 
the  patient  and  determine  what  symptoms  are  present  and  what 
disease  or  conditions  they  indicate.  The  physician  must 
make  a  decision  as  to  what  treatment  to  prescribe.  The 
physician  then  monitors  the  condition  of  the  patient  and 
make  a  series  of  decisions  as  to  whether  to  continue  the 
current  treatment,  change  to  another,  or  simply  stop 
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treatment.  The  decisions  are  not  independent,  that  is 
treatment  prescribed  in  previous  periods  will  affect  the 
current  decision  and  may  have  side-effects  which  induce 
other  symptoms  or  conditions.  Two  medications  may  counter 
each  other's  effects.  The  state  of  the  patient's  health 
changes  autonomously  as  a  result  of  the  disease  and  also  as 
a  result  of  the  treatment.  These  decisions  must  be  made  in 
real  time.  A  physician  cannot  keep  a  dying  patient  waiting 
while  he/she  thinks  the  problem  over  at  length. 

In  a  military  context,  the  Tactical  Action  Officer  (TAO) 
standing  watch  in  the  Combat  Information  Center  (CIC)  of  a 
warship  is  operating  in  a  complex  dynamic  decision  making 
environment.  His  goal  is  to  fight  the  ship,  that  is,  in 
essence  to  destroy  the  enemy,  and  prevent  any  damage  to  his 
own  ship.  He  takes  information  from  ship's  sensors  to 
classify  contacts  as  friendly,  hostile  or  neutral.  He  must 
decide  what  actions  to  take  to  protect  his  ship.  He  can 
attempt  to  warn  them  off,  destroy  them,  interfere  with  their 
2d)ility  to  attack  him  or  simply  keep  a  watchful  eye  on  them. 
The  disposition  of  these  other  forces  will  change 
autonomously  according  to  their  own  operational  plan.  In 
addition,  the  TAO's  actions  will  draw  a  reaction  from  them. 
For  instance,  illuminating  an  aircraft  with  a  fire  control 
radar  is  considered  to  be  a  hostile  act,  and  will  most 
likely  either  cause  the  aircraft  to  withdraw  or  attack. 
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All  of  these  dynamic  decision  situations  embody  four 
common  characteristics  (Brehmer,  1992).  First,  a  series  of 
decisions  is  required  to  reach  the  goal.  Both  the  physician 
and  the  TAO  must  continuously  monitor  the  situation  and 
iteratively  decide  what  action,  if  any,  to  take.  Second, 
the  decisions  are  not  independent.  A  medication  may  produce 
side  effects.  If  the  TAO  ceases  radar  emissions  to  avoid 
detection  by  a  hostile  force,  he  forgoes  the  ability  to 
detect  that  force  beyond  visual  range.  Also  any  missile  he 
fires  at  one  contact  cannot  be  fired  at  another.  Third,  the 
state  of  the  problem  changes,  both  autonomously  and  as  a 
result  of  the  decisions.  Both  a  disease  and  a  hostile  force 
will  progress  on  their  own,  changing  the  situation. 

However,  the  situation  is  also  altered  by  the  actions  cf  the 
physician  or  TAO.  Fourth,  the  decisions  must  be  made  in 
real  time.  Hesitation  in  both  the  medical  and  military 
communities  can  carry  drastic  penalties.  It  is  a  fact  of 
life  that  time  waits  for  no  one. 

Thus,  individuals  make  many  decisions  in  complex  dynamic 
enviroiunents.  What  factors  influence  the  quality  of  these 
decisions  is  therefore  of  importance.  However,  as  Hogarth 
(1981)  and  Brehmer  (1992)  point  out,  much  of  the  research  in 
decision  making  has  been  conducted  in  static  contexts, 
whereas,  individuals  must  operate  in  dynamic  decision 
situations. 
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B.  EZPBRINEIITAL  METHODOLOOT 


1 .  Conceptual  Basis 

Decisions  are  made  not  for  their  own  sake,  but  to 
achieve  control  of  the  system  (Brehmer,  1992),  enabling  some 
desirable  condition  to  be  achieved  or  maintained  (Brehmer, 
1992;  Hogarth,  1981).  Brehmer  (1992,  p.  217)  has 
identified  four  general  conditions  for  control  of  a  system. 

-  there  must  be  a  goal  (the  goal  condition), 

-  it  must  be  possible  to  ascertain  the  state  of  the 
system  (  the  observability  condition), 

-  it  must  be  possible  to  affect  the  state  of  the 
system  (the  action  condition), 

-  th( ^e  must  be  a  model  of  the  system  (the  model 
condition) . 

The  observability  and  action  conditions  are 
properties  of  the  system.  The  goal  and  model  are  properties 
of  the  decision  maker.  Decision  makers  form  their  goals  and 
model  based  on  the  observability  and  action  conditions 
presented  to  them  (Brehmer,  1992).  It  is  this  translation 
of  observability  and  actions  into  goals  and  models  that  is 
of  interest. 

Feedback  is  critical  to  the  formulation  of  mental 
models.  Various  feedback  strategies  may  be  employed.  In 
outcome  feedback  control,  only  information  about  the  current 
situation  is  used  to  determine  an  action.  This  strategy 
works  well  in  the  ed>sence  of  feedback  delays  (Brehmer,  1992; 
Sengupta,  1992).  In  feedforward  control,  a  model  developed 
prior  to  beginning  the  task  is  used  to  predict  the  actual 
state  of  the  system,  and  what  actions  are  required  to 
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achieve  or  maintain  a  desired  state.  This  requires  that  the 
system  must  not  change  significantly  over  time,  because 
changes  would  invalidate  the  model  (Brehmer,  1992;  Sengupta 
and  Abdel-Hamid,  1992). 

The  feedback  strategy  of  the  decision  maker 
determines  the  kind  of  model  needed  (Brehmer,  1992). 

Brehmer  (1992)  continues  on  to  emphasize  the  need  for  an 
experimental  paradigm  that  allows  the  observability  and 
action  possibilities  to  be  defined  in  such  a  way  that  the 
svibject's  goals  and  mental  models  can  be  inferred.  (Brehmer, 
1992) 

2 .  Methodological  Basis 

Field  studies  of  dynamic  decision  making  are 
h£unpered  by  the  complexity  of  most  real  world  systems 
(Brehmer,  1992).  It  is  difficult  if  not  impossible  for  the 
experimenter  to  develop  a  realistic  model  of  the  tasks.  The 
effects  of  environmental  noise  are  difficult  to  gauge.  That 
is,  the  experimenter  is  hcunpered  by  opaqueness  as  much  as 
the  subjects. 

Therefore,  microworlds  have  been  introduced  to  study 
dynamic  decision  making.  A  microworld  is  a  computer  based 
dynamic  simulation  with  which  subjects  interact.  Numerous 
studies  have  already  used  microworlds  of  welfare 
administration  (Mackinnon  and  Wearing,  1980),  medical 
treatment  (Kleinmuntz  and  Kleinrauntz,  1981),  forest  fire 
fighting  (Loevborg  and  Brehmer,  1992;  Brehmer  et  al.,  1992a; 
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Brehmer  et  al.,  1992b). 

An  example  of  a  computer  simulated  microworld  is 
NEWFIRE  (Loevborg  and  Brehmer,  1991),  in  which  subjects  act 
as  the  commander  of  a  forest  fire  fighting  force.  NEWFIRE 
incorporates  the  four  characteristics  of  dynamic  decision 
making.  A  series  of  commands  to  fire  fighting  units 
(decisions)  is  required  to  combat  the  forest  fires,  as  the 
fires  can  rarely  be  extinguished  in  one  time  period  by  one 
unit.  The  decisions  are  interrelated  since  once  a  unit  has 
begun  fighting  a  fire  in  one  location,  it  must  extinguish 
that  fire  completely  before  it  can  be  dispatched  to  another 
area.  The  state  of  the  fire  changes  autonomously  as  a 
result  of  wind  and  the  algorithm  that  propagates  the  fire. 
Because  locations  where  fire  has  been  extinguished  or  burned 
out  on  its  own  cannot  re-ignite,  the  actions  of  the  fire 
fighting  units,  determined  by  the  commander,  also  alter  the 
state  of  the  fire(s).  Decisions  must  be  made  in  real  time 
because  the  NEWFIRE  simulation  is  controlled  by  the 
computer's  clock.  The  simulation  does  not  halt  and  wait  for 
the  commander  to  input  his/her  commands  (Loevborg  and 
Brehmer,  1992). 

NEWFIRE  also  meets  Brehmer's  (1992)  four  general 
conditions  for  control  of  a  system.  The  goal  is  to 
extinguish  the  forest  fires.  Through  reports  from  the 
spotter  plane  and  fire  fighting  units,  the  state  of  the 
system  can  be  ascertained.  The  state  of  the  system  is 
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a£f«'Cted  by  the  ordering  of  units  to  fight  fires.  Subjects 
are  able  to  develop  a  mental  model  of  how  the  fire 
propagates  and  how  the  fire  fighting  units  affect  it. 

C.  PRIOR  RESEARCH 

Several  studies  in  decision  making  have  already  been 
conducted  using  microworlds.  These  have  tended  to  focus  on 
feedback  (ob8erved>ility)  and  the  formation  of  mental  models 
(model)  by  studying  the  actions  of  decision  makers. 
Microworlds  have  been  used  to  simulate  a  forest  fire 
fighting  task  (Loevborg  and  Brehmer,  1991;  Brehmer  et  al., 
1992a;  Brehmer  et  al.,  1992b),  medical  treatment  (Kleinmuntz 
and  Kleinmuntz,  1981;  Kleinmuntz,  1985)  and  welfare 
administration  project  (Mackinnon  and  Wearing,  1980). 
However,  the  results  of  these  experiments  are  generally 
representative  only  of  the  characteristics  of  the  subjects' 
mental  models.  Very  few  conflicting  goals  have  been  present 
in  the  tasks  studied  thus  far  (Brehmer,  1992).  To  that  end, 
Loevborg  and  Brehmer  (1991)  have  introduced  a  version  of 
NEWFIRE,  a  forest  fire  fighting  simulation.  This  is  a 
microworld  which  -'J  lows  the  experimenter  to  use  cost  factors 
to  examine  the  effects  of  multiple  conflicting  goals. 

NEWFIRE 's  usefulness  as  an  experimental  tool  has  been 
shown  in  several  studies  (Brehmer  et  al.,  1992a;  Loevborg 
and  Brehmer,  1992;  Brehmer  et  al.,  1992b).  Brehmer, 

Loevborg  and  Winman  (1992a)  have  used  it  to  investigate  how 
subjects'  learn  the  tim<?  constants  of  a  task.  Marsden  and 
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Reason,  and  later  Inner  and  Reason,  have  studied  subjects' 
initial  adaptation  to  a  task  (Loevborg  and  Brehmer,  1992). 
Brehiner,  Loevborg  and  winman  (1992b)  have  used  NEWFIRE  to 
ez2unine  the  effects  of  costs  in  a  dynamic  decision 
environment.  In  this  latest  study,  goals  were  explicitly 
studied. 

In  two  experiments,  Brehmer,  Loevborg  and  Winman  (1992b) 
have  used  fairly  simple  scenarios  where  the  fire  can  be 
extinguished  quickly  and  with  few  resources.  Each  scenario 
lasted  on  average  about  four  to  six  minutes.  The  test 
scenarios  involved  only  one  ignition,  of  either  a  "close 
fire"  or  a  "distant  fire".  A  "close  fire"  is  one  whose 
ignition  occurs  one  or  two  cells  from  the  initial  location 
of  a  fire  fighting  unit,  and  can  be  extinguished  by  only  one 
unit.  A  "distant  fire"  is  an  ignition  five  or  six  cells 
from  the  initial  location  of  a  fire  fighting  unit,  and  can 
be  extinguished  by  a  minimum  of  four  fire  fighting  units. 

The  number  of  units  mentioned  here  is  optimal.  Delays  in 
the  initial  reaction  to  an  ignition,  or  in  the  case  of  a 
distant  fire  dispatching  too  few  units,  would  allow  the  fire 
to  grow.  More  resources  would  then  have  to  be  used  to  put 
out  the  fire.  (Brehmer  et  al.,  1992b) 

The  first  of  these  two  experiments  involved  three 
experimental  conditions.  The  control  group  was  given  no 
cost  information.  One  experimental  group  was  given 
information  ed>out  costs.  The  other  experimental  group 
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received  cost  information,  and  a  display  of  the  cumulative 
costs  incurred  while  performing  the  fire  fighting  task. 

Both  experimental  groups  were  told  to  fight  the  fires  as 
economically  as  possible,  while  preserving  the  base.  In 
effect  they  were  given  two  goals.  The  results  showed  that 
those  given  a  cumulative  cost  display  performed  worse  than 
those  who  were  only  told  to  consider  cost.  Both 
experimental  groups  performed  worse  than  the  control  group. 
(Brehmer  et  al.,  1992b) 

The  second  experiment  sought  to  determine  if  the  cost 
goal  hindered  performance  or  simply  learning.  To 
distinguish  whether  the  cost  goal  affected  performance  or 
learning,  subjects  learned  the  task  by  practicing  NEWFIRE 
without  any  knowledge  of  costs.  They  were  divided  into  an 
experimental  group  which  was  informed  of  costs,  and  a 
control  group  which  was  not.  The  results  suggested  that  the 
additional  goal  of  minimizing  costs  heunpered  learning,  not 
performance.  That  fear  of  poor  cost  performance  induced 
risk-averse  behavior  is  presumed  to  be  the  cause  of  these 
findings.  (Brehmer,  et  al.,  1992) 

This  thesis  intends  to  further  this  work  examining  how 
subjects  handle  a  number  of  goals.  We  also  increased  the 
complexity  and  difficulty  of  the  tasks,  thereby 
necessitating  a  longer  series  of  decisions.  Thus,  the 
scenarios  employed  here  last  longer.  Subjects  typically 
complete  five  scenarios  in  one  to  one  and  a  half  hours. 
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compared  to  sixteen  scenarios  in  the  same  time  period  in  the 
Brehmer  and  Loevborg  (Brehmer  et  al.,  1992a;  Brehmer  et  al., 
1992b)  experiments.  By  studying  a  longer  duration  of 
dynamic  decisions,  it  is  hoped  that  insight  may  be  gained 
into  some  longer  term  impacts  of  a  series  of  interrelated 
decisions . 

D.  EXPECTED  RESULTS 

It  would  be  expected  that  as  the  number  of  goals  and  the 
task  complexity  increase,  the  quality  of  decisions  will 
decrease.  Individuals  have  severe  limits  on  their  ability 
to  process  information,  and  therefore  complexity  in  the 
environment  should  degrade  the  performance  of  the  individual 
in  decision  making  (MacKinnon  and  Wearing,  1980). 

Task  complexity,  in  NEWFIRE,  is  found  in  the  number  of 
fires,  the  distance  between  the  fire  and  the  nearest  fire 
fighting  unit,  the  efficacies  of  the  fire  fighting  unit  and 
the  direction  and  velocity  of  the  wind,  which  affects  the 
propagation  of  the  fire.  It  is  reasonable  to  expect  that 
increasing  the  complexity  will  decrease  the  guality  of  the 
decision  maker's  performance  (MacKinnon  and  Wearing,  1980). 
One  probably  will  not  fight  two  fires  as  easily  as  one,  and 
the  farther  away  the  fire  the  more  difficult  it  will  be  to 
extinguish.  This  is  because  while  the  units  are  travelling 
to  the  fire,  it  will  have  time  to  spread  and  intensify. 
Similarly,  a  strong  wind  will  fan  the  flames,  spreading  the 
fire  more  quickly. 
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A  greater  number  of  goals  would  also  be  expected  to 
degrade  performance.  With  only  one  goal,  the  decision  maker 
can  work  with  a  model  that  is  biased  toward  that  aspect  of 
the  system.  Since  this  is  a  greatly  simplified  version  of 
reality,  it  is  also  cognitively  easier  to  manipulate, 
thereby  yielding  improved  performance.  But  with  two  goals 
to  consider,  the  model  becomes  more  complex.  Additional 
aspects  of  the  environment  that  pertain  to  the  achievement 
of  the  second  goal  must  now  be  entered  into  the  model.  More 
elements  and  more  interaction  among  the  elements  add  to  the 
complexity  of  the  model.  This  model  is  no  longer  as  simple 
to  use,  and  as  a  consequence  performance  will  suffer. 
Consider  the  decision  of  how  many  fire  fighting  units  to 
dispatch  to  an  ignition.  If  the  only  goal  is  to  minimize 
the  eunount  of  forest  lost  to  fire,  it  would  be  wise  to 
respond  with  more  units  than  actually  believed  necessary. 

The  additional  units  provide  a  safety  margin  with  little  or 
no  detrimental  effect  on  achieving  the  goal.  But  if  costs 
are  also  to  be  minimized,  then  the  decision  maker  is 
penalized  for  sending  any  more  units  than  are  actually 
needed.  The  tendency  in  this  second  case,  with  two  goals  is 
to  not  dispatch  a  unit  unless  it's  presence  is  vital  to 
extinguishing  the  blaze  (Loevborg  and  Brehroer,  1992). 

Besides  the  added  complexity,  multiple  goals  may 
sometimes  conflict  with  each  other.  To  do  well  in  pursuit 
of  one  goal  may  require  sacrificing  progress  toward  another 
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goal.  For  instance,  consider  a  fire  that  is  completely 
encircled  by  ash,  but  is  still  burning  inside.  With  only 
one  goal  of  minimizing  area  lost  to  fire,  the  decision  to 
send  units  into  the  fire  to  preserve  what  little  forest 
remains  is  fairly  straight  forward.  However  if  the  decision 
maker  is  also  attempting  to  minimize  costs  (conserve 
resources)  the  decision  is  not  at  all  simple.  The  fire 
cannot  spread  beyond  the  surrounding  wall  ash,  so  the 
decision  to  attack  the  fire,  while  reducing  potential  lost 
area,  increases  costs.  On  the  other  hand,  not  incurring  the 
cost  of  fighting  that  fire  means  more  area  burned  (Brehmer 
et  al. ,  1992b) . 

People  would  be  expected,  due  to  limits  of  cognitive 
abilities,  to  concentrate  on  achievable  goals,  to  the  extent 
they  are  not  failing  too  terribly  in  achieving  another  goal. 
It  is  expected  that  some  goal  shifting  would  occur.  That 
is,  when  a  subject  realizes  he/she  is  progressing  poorly 
toward  one  goal,  (for  instance,  minimizing  the  amount  of 
forest  burned)  while  still  within  reach  of  another  goal 
(minimizing  costs),  they  will  make  decisions  that  favor  the 
first  goal  over  the  latter.  Once  (re) gaining  control  of 
progress  towards  the  second  goal,  subjects  would  likely 
continue  paying  closer  attention  to  that  goal,  until  it 
appears  that  the  first  goal  will  not  be  met.  Thus  subjects 
are  expected  to  shift  attention  between  goals,  but  generally 
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diminishing  prospects  of 


III.  METHODOLOGY 


A.  TASK  EMVIROHMEHT 

NEWFIRE  (Loevborg  and  Brehmer,  1991)  is  a  microworld 
designed  to  simulate  a  forest-fire  fighting  environment. 
Loevborg  and  Brehmer  (1991)  describe  the  NEWFIRE 
enviroiunent.  Figure  1  shows  the  NEWFIRE  screen.  The  forest 
area  is  represented  to  the  left  by  a  large  green  square, 
divided  into  an  18  x  18  grid.  Each  grid  coordinate  can  be 
identified  by  column,  numbered  1  through  18  from  left  to 
right  (East  to  West),  and  row,  lettered  A  through  R  from  top 
to  bottom  (North  to  South).  Each  cell  represents  1  hectare 
of  forest.  The  base  camp  appears  as  a  brown  cell  and  is 
denoted  by  a  'B*.  Eight  individual  fire  fighting  units 
(FFU)  appear  as  light  blue  numbered  squares.  A  fire  appears 
as  a  red  cell(s)  in  the  forest  grid. 

The  subject,  acting  as  commander  of  the  eight  fire 
fighting  units,  orders  them  to  the  fire.  The  icon  for  an 
individual  fire  fighting  unit  changes  color  to  suit  its 
task.  While  inactive,  or  watching  for  a  fire  the  icon  is 
light  blue.  When  the  unit  begins  travelling  to  its  ordered 
destination,  it  changes  color  to  purple,  and  a  small  purple 
number  corresponding  to  that  unit  appears  in  the  upper  left 
corner  of  the  destination  cell.  The  fire  fighting  unit 
turns  dark  blue  when  engaged  in  a  fire  fighting  operation. 
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The  activity  information  and  current  location  of 
each  fire  fighting  unit  is  displayed  in  brief  text  in  the 
message  panel, 

the  middle  window  on  the  right  hand  side  of  the  screen.  The 
lower  right  hand  panel  displays  each  fire  fighting  unit's 
number  and  its  ordered  destination,  if  any. 

The  wi  .d  direction  and  velocity  are  displayed  in  the 
upper  right  hand  corner.  The  arrow  points  downwind,  and  the 
numbers  around  the  compass  are  reversed,  so  that  a  heading 
indicated  by  the  arrow  head  is  the  direction  from  which  the 
wind  blows.  Wind  velocity  is  given  as  a  number  in  meters 
per  second. 


Ciunulative  time,  forest  area  burned  and  costs  (if 
applicable)  are  displayed  in  the  banner  across  the  top  of 
the  screen. 

Each  fire  fighting  unit  operates  semi-autonomously. 

They  rely  on  the  commander  to  deploy  them,  but  once  given  an 
order  they  will  carry  it  out  without  any  further  guidance. 
All  units  begin  the  simulation  inactive,  and  become  active 
when  given  an  order  to  move.  They  travel  in  a  straight-line 
to  their  destination.  The  cells  are  large  enough  that  they 
can  safely  traverse  through  a  burning  cell.  An  fire 
fighting  unit  can  be  given  a  destination  at  any  time.  If  it 
is  inactive  or  watching,  it  will  immediately  begin 
travelling  to  the  destination.  If  it  is  already  travelling, 
it  ignores  the  previous  destination  and  begins  travelling 
immediately  to  the  most  recent  destination.  If  the  unit  is 
engaged  in  a  fire  fighting  operation  it  will  finish  that 
operation  but,  upon  completion,  will  automatically  travel  to 
the  destination  it  was  most  recently  given. 

When  a  fire  fighting  unit  arrives  at  its  destination, 
its  actions  are  determined  by  the  state  of  that  cell.  If 
the  cell  is  unburned  forest,  the  unit  watches  for  a  fire. 

If  the  cell  is  burnt  forest,  the  unit  perceives  this  as  a 
parking  maneuver  and  becomes  inactive.  If  the  cell  is 
burning,  the  unit  begins  its  fire  fighting  operation.  The 
first  step  is  mobilizing,  that  is,  it  unpacks  its  equipment 
from  the  vehicle.  It  then  fights  the  fire,  and  when  the 
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fire  has  been  extinguished,  it  demobilizes,  packing  its 
equipment  in  preparation  for  its  next  move.  If  the  fire 
fighting  unit  has  not  received  an  order  by  the  time  it 
finishes  demobilizing,  it  will  remain  in  place,  becoming 
inactive. 

B.  SIMULATION  MODEL 

The  spread  of  the  fire  is  controlled  by  a  cellular 
automaton.  Left  alone,  the  fire  in  a  cell  will  burn  itself 
out  15  time  units  after  catching  fire.  Within  each  cell, 
the  intensity  of  the  fire  increases  linearly  to  time  7.5  and 
then  decreases  linearly  after  that.  Since  the  process  of 
fighting  the  fire  is  itself  also  linear,  a  fire  fighting 
unit  will  extinguish  a  **young”  or  "old”  fire  in  appreciably 
less  time  than  a  "middle-aged"  fire.  (Loevborg  and  Brehmer, 
1991)  This  is  illustrated  in  Figure  2  (Loevborg  and 
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The  fire  spreads  throughout  the  forest  in  the  shape  of 
an  ellipse  with  an  eucis  parallel  to  the  wind.  The  higher 
the  velocity  of  the  wind,  the  more  narrow  and  elongated  will 
be  the  fire's  footprint.  (Loevborg  and  Brehmer,  1991) 

Once  the  fire  in  a  particular  cell  has  been  extinguished 
or  burned  out  on  its  own,  that  cell  cannot  re-ignite,  even 
if  only  a  very  small  portion  of  the  cell  has  actually 
burned.  Thus,  by  fighting  only  the  leading  edge  of  the 
fire,  a  wall  can  be  created  to  prevent  further  spread  of  the 
fire.  (Loevborg  and  Brehmer,  1991)  This  characteristic  can 
be  very  helpful  to  the  subject,  but  can  also  be  a  hindrance. 
As  Ketscher  (1992)  noted,  the  elliptical  pattern  of  the 
fire's  spread  is  obscured,  and  interferes  with  the  subjects' 
^d)ility  to  form  a  mental  model  of  fire  propagation. 

C.  EXPERINBMTAL  DESIGN 

The  research  design  is  illustrated  in  Table  1.  The 
experiment  used  a  factorial  design  with  two  components  to 
capture  the  nutmber  of  objectives  and  the  task  difficulty. 

The  number  of  objectives  was  operationalized  as  a  between 
subjects  condition  and  the  task  difficulty  as  a  within 
subjects  condition. 
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Table  1.  EXPERIMENTAL  DESIGN.  Number  of  subjects  per 


group. 

Order  of 
Scenarios 

Number  of  Objectives  Group  | 

One 

Two 

Three 

Order  1 
(D,  M,  E) 

9 

9 

9 

1  Order  2 

1  (E,  D,  M) 

9 

9 

9 

1  Order  3 

1  (M,  E,  D) 

9 

9 

9 

Randomization  of  within  subjects  conditions  was  achieved 
using  a  Latin  Squares  design  as  follows  (Kirk,  1982,  pp. 
311-312) : 

First,  each  task  was  assigned  a  corresponding  letter. 

E :  Easy 
M:  Moderate 
D:  Difficult 

Next,  two  sequences  of  random  numbers  were  generated. 

(3,  1,  2) 

(1/  3,  2) 


The  square  is  then  selected. 

E  M  D 
M  D  E 
DEM 
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Rows  are  then  arranged  according  to  the  first  set  of 

random  numbers. 

DEM 
E  M  D 

M  D  E 

Columns  are  then  arranged  according  to  the  second  set  of 

random  numbers. 

D  M  E 

EDM 
MED 

Finally  each  group  is  assigned  to  one  these  orders  as 
below: 

Group  1:  D,  M,  E 
Group  2:  E,  D,  M 
Group  3:  M,  E,  D 

Thus,  Group  1  conducts  tasks  in  the  order:  Difficult, 
Moderate  and  Easy. 

1 .  Between-Sub j  ects 

The  fundamental  objective  of  this  experiment  is  to 
determine  the  effect  of  varying  levels  of  system  complexity 
on  decision  making  in  dynamic  environments.  The  goals  given 
to  subjects  constituted  the  three  between-subjects 
conditions.  The  predominant  goal  of  all  groups  was  to 
preserve  the  base.  One  group  was  told  only  to  minimize  the 
area  of  forest  buriisd  by  the  fire.  The  second  group  was 
given  two  goals,  to  minimize  the  area  burned  and  to  minimize 
the  costs  incurred  while  fighting  the  fire.  The  third  group 
was  given  three  goals,  minimize  the  area  burned,  costs 
incurred  and  the  time  taken  to  extinguish  the  fire.  The 
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order  that  the  objectives  were  presented  to  subjects  was 
randomized  within  the  two  and  three  objective  groups. 


2.  Nithln-Subjects 

In  addition,  the  experiment  sought  to  capture  the 
effects  within  each  condition  over  time.  In  this  experiment 
each  subject  performed  three  scenarios,  an  easy,  a  moderate 
and  a  difficult  one.  The  differences  between  the  scenarios 
are  summarized  in  Table  2.  The  easy  scenario  involved  only 
one  ignition.  Wind  velocity  and  direction  were  held 
constant  throughout.  All  fire  fighting  units  were 
homogeneous;  that  is,  every  unit  had  the  seune  moving  speed 
and  put-out  speed  as  every  other  unit.  In  the  moderate 
scenario,  a  second  fire  was  ignited  at  time  6  in  a  cell 
relatively  close  to  the  first  ignition.  Wind  direction 
remained  constant,  but  the  speed  changed.  All  fire  fighting 
units  were  homogeneous.  In  the  difficult  scenario,  a  second 
fire  was  ignited  at  time  6  in  a  cell  distant  from  the  first 
ignition.  Both  wind  direction  and  velocity  vary 


TABLE  2.  SCENARIO  SUMMARY 


[ 

EASY 

MODERATE 

DIFFICULT 

1  No. 

of 

1 

2 

2 

1  ignitions 

1  ^ 

Direction 

Constant 

Constant 

Changes 

h 

(000) 

(270) 

(135  to  045) 

Velocity 

Constant 

Changes 

Changes 

(10  m/s) 

(18  to  6  m/s) 

(4  to  18  m/s) 

1  Homogeneous 

Yes 

Yes 

No 

1  FFU 

•s  ? 

significantly,  but  only  once.  Four  of  the  fire  fighting 
units  are  half  as  effective  (in  both  moving  speed  and  put- 
out  speed)  as  the  other  four. 

Thus,  the  moderate  scenario  is  more  complex  than  the 
easy  scenario  by  increasing  the  number  of  separate  ignitions 
from  one  to  two,  and  by  reducing  wind  velocity  during  the 
fire  fighting.  The  complexity  added  by  the  second  ignition 
is  obvious.  By  reducing  the  wind  velocity  but  not 
direction,  the  pattern  of  fire  propagation  changes  from  a 
narrow  ellipse  stretched  in  the  direction  of  the  wind  to 
more  rounded,  almost  circular  shape. 

The  difficult  scenario  is  made  progressively  more 
complex  by  changing  both  wind  direction  and  velocity,  and  by 
the  fact  that  the  fire  fighting  units  are  not  of  equal 
abilities.  Shifting  the  wind  direction  changes  the 
direction  in  which  the  fire  tends  to  spread.  The  increase 
in  wind  velocity  narrows  the  elliptical  pattern  of  the  fire 
and  increases  the  speed  with  which  it  spreads  to  neighboring 
downwind  cells.  Only  four  of  the  eight  fire  fighting  units 
are  as  effective,  in  terms  of  moving  speed  and  put  out 
speed,  as  the  units  in  the  easy  and  moderate  scenarios.  The 
other  four  units  are  half  as  effective.  These  travel  only 
half  as  fast  and  put  out  fires  at  half  the  rate  of  the  more 
effective  units. 
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3.  Subjects 

The  experiment  was  conducted  using  81  graduate 
students.  Participant  profiles  are  Contained  in  Table  3. 
Participants  were  divided  into  nine  cells  based  on  objective 
functions  and  task  order. 

A  random  stream  of  two  digit  numbers  was  generated 
using  a  random  number  teUsle.  Duplicates  and  numbers  greater 
than  81  were  disregarded.  Each  svibject  was  assigned  the 
subsequent  number  in  that  stream,  based  on  the  order  he/she 
arrived.  Random  arrival  of  subjects  was  assumed. 

D.  EXPERIMEMTAL  COEDUCT 

The  experiment  was  conducted  in  a  computer  lab  with  the 
same  attendant  present  for  the  duration  of  all  trials. 
Subjects  were  given  time  to  read  the  documentation. 

Subjects'  questions  regarding  manipulation  of  the  interface, 
asked  either  before  or  during  the  trial  were  answered. 
Questions  pertaining  to  strategy  in  performing  the  task  were 
not  permitted. 


TABLE  3.  Participant  Profiles  (Means) 
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The  simulation  was  run  in  a  computer  leU:>  using  a  single 
personal  computer,  with  a  387  math  co-processor.  Only  one 
siibject  performed  the  experiment  at  one  time. 

All  subjects  performed  a  mouse  practice  exercise,  and 
one  or  two  practice  scenarios  before  beginning  the  actual 
experiment,  where  they  received  different  types  of 
information  concerning  their  goals.  The  steps  performed  by 
each  group  are  summarized  in  Table  4. 

1.  House  Practice 

The  purpose  of  the  mouse  practice  facility  in 
NEWFIRE  is  to  ensure  all  subjects  possess  a  requisite  level 
of  dexterity  with  the  mouse,  thereby  eliminating  the  noise 
of  mouse  manipulation  difficulties  from  the  experiment 
result  (Loevborg  and  Brehmer,  1992).  These  difficulties 
arise  from  one  of  two  sources.  First,  some  subjects  rarely 
or  never  use  a  mouse.  Second,  the  "speed"  of  the  mouse, 
i.e.  ratio  of  mouse  movement  to  cursor  movement,  is  not 
often  different  from  that  which  experienced  mouse  users  are 
accustomed  to  on  their  own  computers. 

Operating  instructions  for  the  mouse  practice 
facility  were  read  by  the  subject,  and  any  questions  were 
answered.  These  instructions  are  provided  in  Appendix  B. 

The  mouse  practice  facility  presents  the  s\ibject  with  two 
grids.  Blue  icons  appear  at  varying  intervals  in  random 
squares  in  the  grid  to  the  left.  The  subject  uses  the  mouse 
to  move  those  icons  to  the  grid  on  the  right. 


2 .  Practlc*  Scenarios 

After  completion  of  the  mouse  practice,  subjects 
read  the  instructions  for  the  NEWFIRE  environment,  included 
in  Appendix  C.  The  first  twenty-five  subjects  then 
performed  one  practice  scenario,  described  in  Appendix  A, 
before  beginning  the  experiment.  This  one  practice  did  not 
provide  subjects  with  sufficient  opportunity  to  learn  how  to 
operate  NEWFIRE  and  develop  a  mental  model  of  the  task.  The 
remaining  subjects  therefore  also  performed  an  additional, 
more  difficult  practice  scenario,  described  in  Appendix  A. 

3 .  Experiment  Scenarios 

Information  on  goals  was  provided  to  subjects  as 
appropriate  to  their  experimental  conditions.  Subjects 
receiving  cost  information  also  received  the  budget 
information  for  each  task  immediately  before  beginning  that 
task.  These  documents  are  shown  in  Appendix  D.  Each 
subject  performed  three  tasks,  detailed  in  Appendix  A. 

4 .  Questionnaire 

After  completing  the  experiment,  subjects  filled  a 
questionnaire,  appropriate  to  their  between  subjects 
condition.  All  three  questionnaires  contained  the  same  core 
questions,  but  groups  receiving  two  objectives  and  three 
objectives  were  asked  additional  questions  pertaining 
specifically  to  cost  and  other  goals.  The  questions  asked 
are  contained  in  Appendix  E. 
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TABLE  4.  Sequence  of  Experiment. 


Siibjects* 

Actions 

1  Order  of  Actions 

One 

Objective 

Group 

Two 

Objectives 

Group 

Three 

Objectives 

Group 

Read  mouse 
instructions 

1 

1 

1 

Perform  mouse 
practice 

2 

2 

2 

Read  NEWFIRE 
instructions 

3 

3 

3 

Perform  practice 
scenarios 

4 

4 

4 

Read  Cost 

Information 

N/A 

5 

5 

Read  first 
scenario  budget 

N/A 

6 

6 

information 

Perform  first 
scenario 

5 

7 

7 

Read  second 
scenario  budget 
information 

N/A 

8 

8 

Perform  second 
scenario 

6 

9 

9 

Read  third 
scenario  budget 
information 

N/A 

10 

10 

Perform  third 
scenario 

7 

11 

11 

Fill  out 
questionnaire 

8 

12 

12 
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B.  RESULTS 


The  experiment  utilized  81  participants.  The  results  of 
nine  subjects  were  discarded  for  the  following  reasons.  Two 
subjects  failed  to  complete  the  scenarios  due  to  a  hardware 
malfunction.  Four  subjects  lost  the  base,  and  therefore 
their  results  were  discarded.  Two  subjects'  performances 
exceeded  more  than  two  standard  deviations  from  the  mean. 
Results  of  one  subject  were  discarded  due  to  a  procedural 
error  by  the  experimenter.  Therefore  the  results  of  72 
subjects  were  analyzed.  Figures  3  through  5  show  the 
performance  between  subjects  and  within  subjects,  measured 
by  cost,  area  and  time  burned. 


Figure  4.  Mean  Areas  Burned 

The  hypotheses  were  tested  by  applying  the  following 
analysis  of  variance  (ANOVA)  model  for  multiple  Latin 
Squares,  as  suggested  by  Winer  (1971,  p.697): 


lOtiMlM 


Figure  5.  Mean  Times 
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Y^jin  =  /i  +  +  (afl).j  +  where: 

H  is  constant, 

is  the  number  of  objectives  group  (i  =  i,  2,  3,  where  1  = 
one  objective,  2  =  two  objectives,  and  3  =  thre'^i 
objectives) , 

B^  is  the  type  of  scenario  (j  =  1,  2,  3,  where  1  *  easy,  2  = 

moderate,  and  3  =  difficult), 

e^j^  is  the  experimental  error  term. 


TABLE  5.  ANALYSIS  OF  VARIANCE.  Dependent  Variable:  Cost. 


Source  of 
Variance 

S.  S. 

Degrees 

of 

Freedom 

F-Value 

P 

R-Square 

Model 

33610699.750 

8 

16.22 

0.0001 

0.385373 

1  Group 

3268012.333 

2 

6.31 

0.0022 

1  Scenario 

28101940.750 

2 

54.26 

0.0001 

Group* 

Scenario 

2240746.667 

4 

2.16 

0.0744 

Subjects 

53605286.083 

207 

HHPj 

The  analysis  followed  the  General  Linear  Models  procedure 
(SAS,  1987).  The  ANOVA  results,  contained  in  Tcdales  5 
through  7,  reveal  that  the  performance  of  subjects  across 
the  three  different  objective  conditions  was  significantly 
different  for  the  three  dependent  variables,  cost  (F=6.31, 
p=0.0001),  area  (F=7.36,  p=0.0008)  and  time  (F*8.45, 
p=0.0003).  Therefore  the  null  hypothesis  of  no  significant 
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TABLE  6.  ANALYSIS  OF  VARIANCE.  Dependent  Variable:  Area. 


Source  of 
Variance 

S.  S. 

Degrees 

of 

Freedom 

F- 

Value 

P 

R-Square 

Model 

120649.19467 

8 

7.17 

0.0001 

0.216869 

Group 

30976.548548 

2 

7.36 

0.0008 

1  Scenario 

71050.695626 

2 

16.88 

0.0001 

1  Group* 

1  Scenario 

18621.950494 

4 

2.21 

0.0689 

I  Subjects 

435673.74859 

207 

differences  among  number  of  objectives  condit j  on  is 
rejected.  The  results  indicate  that  the  subjects' 
performance  was  influenced  by  the  number  of  objectives  they 
attempted  to  satisfy. 

Additionally,  Tables  5  through  7  show  that  the 
performance  within  subjects  was  significantly  different 
depending  on  the  complexity  of  the  scenario  they  faced  in 


TABLE  7.  ANALYSIS  OF  VARIANCE.  Dependent  Variable:  Time. 


Source  of 
Variance 

S.  S. 

Degrees 

of 

Freedom 

F-Value 

P 

R-Square 

Model 

70719.148148 

8 

12.42 

0.0001 

0.324311 

Group 

12036.148148 

2 

8.45 

0.0003 

Scenario 

52207.814815 

2 

36.67 

0.0001 

Group* 

Scenario 

6475.185185 

4 

2.27 

0.0625 

1  Subjects 

147340.291667 

207 
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terms  of  cost  (F-54.26,  p=0.0001),  area  (F=16.88,  p=0.0001) 
and  time  (F=36.67,  p=0.0001).  Therefore  the  null  hypothesis 
of  no  significant  differences  in  performance  depending  on 
the  complexity  of  the  scenario  is  also  rejected. 

Posterior  tests  using  Scheffe's  method  confirm  that  for 
the  three  dependent  variables,  there  were  significant 
differences  between  the  groups  receiving  different  numbers 
of  objectives.  The  mean  values  of  cost,  area,  and  time  of 
the  group  given  one  objective  were  significantly  lower  than 
the  values  for  the  groups  given  two  or  three  objectives 
(p<0.05).  This  supports  the  first  hypothesis  that  subjects 
given  only  one  objective  to  accomplish  will  perform  better. 
However,  there  was  no  significant  difference  in  performance 
between  the  group  receiving  two  objectives  and  the  group 
receiving  three  objectives. 

Additionally,  posterior  tests  show  that  for  the  three 
dependent  variables  significant  differences  of  performance 
exist  between  scenarios  of  different  levels  of  complexity. 

In  terms  of  cost,  area,  and  time,  subjects  performed 
significantly  worse  on  the  most  complex  scenario  than  on  the 
easy  or  moderate  (p<0.05). 


IV.  COHCLUSIOirS 


A.  8UMNART  OF  RESULTS 

The  aim  of  this  study  was  to  investigate  the  effects  of 
multiple  objectives  on  the  quality  of  decision  making  in 
complex  dyneunic  environments  under  different  levels  of  task 
complexity.  NEWFIRE,  a  microworld  simulating  a  forest  fire 
fighting  task  was  used.  Chapter  II  described  in  general 
terms  some  of  the  research  that  has  been  done  with 
microworlds  (see  also  Funke,  1991),  and  pointed  out  that  to 
date  few  studies  have  examined  the  effects  of  multiple,  and 
possibly  conflicting,  objectives  in  a  dynamic  environment. 
Additionally,  Chapter  II  explained  that  more  goals  create 
greater  complexity  in  the  system,  which  is  cognitively  more 
difficult  for  decision  makers  to  manage.  Thus  one  would 
expect  that  a  larger  number  of  goals  assigned  to  a  person 
will  decrease  the  quality  of  decisions  that  person  makes. 
Chapter  II  also  discussed  complexity  arising  from  sources 
other  than  goals.  This  environmental  complexity  would  also 
be  expected  to  decrease  the  quality  of  decisions. 

The  results  of  this  study  support  the  hypotheses. 
Subjects  given  only  one  goal  significantly  outperformed 
those  given  two  and  three  goals,  and  subjects  performed 
better  in  the  easy  scenario  than  they  did  in  the  moderate 
and  difficult  scenarios. 
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B.  BMBLXSIS  OP  RESULTS 

The  results  of  this  experiment  clearly  indicate  that 
persons  attempting  to  achieve  only  one  goal  perform  better 
than  those  attempting  to  satisfy  multiple  goals.  In  Figures 
3  through  5,  the  performance  of  the  two  goal  group  parallels 
the  performance  of  the  one  goal  group.  This  is  also 
supported  by  the  Scheffe  posterior  tests.  However,  the 
performance  of  the  three  goal  group  is  puzzling.  This  group 
significantly  outperformed  the  two  goal  group  on  the  easy 
and  moderate  scenarios,  but  in  the  difficult  scenario, 
performed  worse  than  either  other  group. 

One  possible  explanation  for  the  anomalous  behavior  of 
the  three  goal  group  is  the  effects  of  goal  shifting.  It 
may  be  that  the  subjects  in  the  three  goal  group  adopted  a 
strategy  (perhaps  unwittingly)  of  continually  and  rapidly 
shifting  among  goals,  in  effect  sampling  their  progress. 

This  pathology  of  decision  making,  termed  'thematic 
vagabonding'  by  Brehmer  (1992),  is  evidenced  by  subjects 
shifting  from  one  part  of  a  problem  to  another,  failing  to 
adapt  to  the  interdependencies  in  the  environment.  It  is 
possible  that  this  strategy  is  as  effective  in  fairly  simple 
tasks  as  concentrating  continually  on  only  one  goal. 

However,  it  is  possible  that  once  task  complexity  increases 
beyond  a  threshold,  the  strategy  of  rapid  and  continuous 
goal  shifting  fails. 

There  was  no  significant  difference  between  subjects' 
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performance  in  the  easy  scenario  and  their  performance  in 
the  moderate  scenario.  Indeed,  subjects  performed  slightly 
better  on  the  moderate  scenario  than  on  the  easy  one.  The 
most  likely  cause  of  this  lies  in  the  difficulties  that 
subjects  have  forming  a  mental  model  of  the  fire 
propagation.  Unless  the  wind  velocity  is  high  (typically  18 
meters  per  second),  the  fire  propagates  upwind,  albeit  at  a 
much  slower  rate  than  its  downwind  propagation.  Subjects 
are  often  not  prepared  for  this  upwind  movement  of  fire. 

This  usually  was  not  a  problem  in  the  moderate  scenario 
because  the  initial  wind  speed  is  18  meters  per  second,  and 
thus  does  not  spread  upwind.  By  the  time  the  wind  velocity 
lessens,  the  upwind  edge  of  the  fire  borders  on  burned 
cells,  which  cannot  re-ignite. 

C.  LIMITING  FACTORS  TO  GENERALI Z ABILITY 

Is  it  reasonedsle  to  extend  these  findings  to  the  real 
world?  This  question  has  two  aspects,  the  use  of  graduate 
students  as  surrogates  for  decision  makers  and  the 
applicability  of  the  task  environment. 

All  of  the  graduate  students  participating  in  this 
experiment  are  also  experienced  military  officers.  Few  of 
the  subjects  have  had  any  previous  fire  fighting  experience. 
They  are  practiced  in  making  decisions  in  their  job 
specialties,  such  flying  aircraft,  driving  ships,  leading  a 
company  of  infantry,  etc.  It  can  reasonably  be  inferred 
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that  the  subjects  had  little  epistemic  competence  for  the 
NEWFIRE  task,  but  did  possess  adequate  heuristic  competence. 
It  is  therefore  reasonedsle  to  assume  their  behavior  in  this 
experimental  investigation  is  an  adequate  representation  of 
their  real  world  decision  making. 

Microworlds,  such  as  MEWFIRE,  have  been  used  in  numerous 
studies  of  aspects  of  cognitive  heurisitics  and  dyneuaic 
decision  making  (Funke,  1991).  Brehmer  (1992)  notes  that 
experiments  with  microworlds  have  yielded  some  consistent 
results  and  that  some  of  the  results  have  been  supported  by 
the  findings  of  field  studies. 


D.  FURTHER  RESEARCH 

Based  on  the  discussion  of  Section  B  above,  future 
research  is  warranted  to  study  the  anomalies  noted  in  the 
behavior  of  the  three  objective  group.  The  experiment  could 
be  replicated  using  the  preservation  of  the  base  as  a  third 
objective  vice  time.  The  scenarios  could  also  be  altered  to 
take  better  advantage  of  the  complexity  the  wind  adds  to  the 
environment.  NEWFIRE  also  allows  for  the  inclusion  of 
meadow,  which  does  not  burn,  in  the  forest  pattern.  By 
placing  ignitions  in  areas  partially  or  totally  enclosed  by 
meadow,  the  trade-offs  between  area  and  cost  become  more 
evident . 

Subjects  in  this  experiment  were  given  no  information 
ed^out  the  model  of  fire  propagation,  and  had  to  develop 
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their  own  model  as  they  performed  the  task.  By  providing 
this  information,  one  might  be  able  to  separate  the  decision 
making  process  from  the  process  of  forming  mental  models.  In 
other  words,  one  might  be  able  to  distinguish  between 
pathologies  of  decision  making  and  pathologies  of  learning. 
This  information  might  be  provided  in  two  ways.  First  by 
simply  including  an  explanation  in  the  instructions. 

Second,  NEWFIRE  has  a  facility  that  allows  a  completed 
scenario  to  be  played  back  on  the  screen  at  an  accelerated 
speed.  It  is  therefore  possible  to  show  subjects  examples 
of  successful  and  unsuccessful  strategies  before  they  begin 
their  trials. 

Subjects  were  not  told  which  fire  fighting  units  were 
less  effective  in  tne  difficult  scenario.  As  a  result,  many 
simply  ignored  the  difference  and  acted  as  if  all  were  still 
homogeneous . 

Other  proposals  for  further  research  require  some 
adaptations  to  NEWFIRE,  with  the  requisite  permission  of 
Loevborg.  Costs  and  wind  velocity  are  currently  displayed 
only  as  numbers.  However,  subjects  tend  to  rely  on  iconic 
information  and  ignore  text  messages  (Ketscher,  1992).  By 
displaying  costs  in  graphic  form,  say  as  a  bar  graph,  and 
altering  the  length  of  the  wind  indicating  arrow  according 
to  wind  velocity,  subjects  might  use  that  information  as 
heavily  as  information  already  presented  iconically. 
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Cost  and  area  are  difficult  to  balance  against  one 
another  because  they  are  measured  in  different  units 
(dollars  vs.  hectares).  By  assigning  dollar  values  to 
forest  area  and  displaying  the  value  of  forest  burned  as 
well  as  area,  subjects'  decision  making  processes  may  become 
more  clear.  This  could  perhaps  be  made  more  sophisticated 
by  varying  the  value  of  individual  cells.  For  instance,  a 
populated  cell  would  be  considered  more  valuedsle  than  an 
unpopulated  one. 

Given  that  dynamic  decision  environments  are  ubiquitous, 
the  implications  of  how  decision  makers  behave  when  faced 
with  a  series  of  interrelated  decisions  are  equally 
widespread.  Situations  confronting  decision  makers  rarely 
involve  only  one  goal.  The  attainment  of  one  goal  often  is 
interlaced  with  the  attainment  of  other  goals.  For 
instance,  for  a  commander  to  successfully  attain  the  goal  of 
landing  an  eunphibious  force,  other  goals  must  be  attained, 
such  as  gaining  air  superiority  and  providing  timely, 
accurate  close  ground  support.  These  goals  themselves  may 
conflict.  Does  one  devote  resources  to  a  strike  against  an 
airfield  well  inland,  or  use  them  against  ground  forces 
closer  to  the  landing  zone?  How  decision  makers  cope  with 
these  multiple  goals  and  the  complex  dynamic  environments  in 
which  they  present  themselves  is  an  area  of  vital  interest. 
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APPBVDIX  A 


DETAILED  DESCRIPTIOM  OP  SCEHARIOS 


AJ  First  practice  scenario 

Uniform  forest 
Base  Location  HIO 
Constant  Wind 

Initial  Wind  315  9  4  m/s 

First  Ignition  location  H6  at  time  T  =  0 
Age  of  fire  when  reported  =  0 
Cost  Factors 

Watching  1 . 00 
Moving  2.00 

Mobilizing,  Demobilizing,  Fighting  4.00 
Clock  Period  »  10  seconds  between  updates 


Fire  Fighting  Unit  Number  | 

1 

2 

3 

4 

5 

6 

7 

8 

Starting 

Position 

G9 

GIO 

Gll 

H9 

Hll 

19 

110 

Ill 

Moving 

Speed 

PH 

1 

PH 

PH 

1 

m 

Put-out 

Speed 

n 

■ 

n 

■ 

■ 

■ 

n 

■ 

A2  Second  practice  scenario 

Uniform  forest 
Base  Location  G14 
Constant  wind 

Initial  wind  270  @  8  ra/s 

First  Ignition  location  J7  at  time  T  =  0 
Age  of  fire  when  reported  =  0 
Cost  Factors 

Watching  1 . 00 
Moving  2.00 

Mobilizing,  Demobilizing,  Fighting  4.00 
Clock  Period  -  10  seconds  between  updates 
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Fire  Fiahting  Unit  Number 


Starting 

Position 


Moving 

Speed 


Put-out 

Speed 


F13 


1.2 


0.5 


F14 


1.2 


0.5 


F15 


1.2 


0.5 


4 


G13 


1.2 


0.5 


615 


1.2 


0.5 


H13 


1.2 


0.5 


H14 


1.2 


0.5 


H15 


1.2 


0.5 


A3  Easy  scenario 

Uniform  forest 
Base  Location  Ml 4 
Constant  wind 

Initial  wind  180  §  10  m/s 
First  Ignition  location  M6  at  time  T  =  0 
Age  of  fire  when  reported  =  0 
Cost  Factors 

Watching  1.00 
Moving  2 . 00 

Mobilizing,  Demobilizing,  Fighting  4.00 
Clock  Period  =  10  seconds  between  updates 


Fire  Fighting  Unit 

(lumber  { 

1 

2 

3 

4 

5 

6 

7 

8 

Starting 

Position 

L13 

L14 

L15 

M13 

M15 

N13 

N14 

N15 

Moving 

Speed 

m 

■ 

n 

m 

■ 

■ 

2.0 

■ 

Put-out 

Speed 

m 

m 

■ 

0.5 

■ 

■ 

m 

■ 

A4  Moderate  scenario 


Uniform  forest 
Base  Location  N5 
Variable  wind 

Initial  wind  090  @  18  m/s 

New  Wind  091  @  6  m/s 

Time  of  Shift  T  =  8 
Rate  of  Shift  2 

First  Ignition  location  HIO  at  time  T  =  0 
Second  Ignition  location  K12  at  time  T  =  5 
Age  of  fire  when  reported  =  0 
Cost  Factors 

Watching  1 . 00 

Moving  2 . 00 

Mobilizing,  Demobilizing,  Fighting  4.00 
Clock  Period  =  10  seconds  between  updates 


"  i 

Fire  Fighting  Unit  Number  | 

1 

2 

3 

4 

5 

6 

7 

8  1 

Starting 

Position 

M4 

M5 

M6 

N4 

N6 

04 

05 

06  1 

Moving 

Speed 

m 

■ 

m 

2.0 

2.0 

2.0 

m 

■ 

Put-out 

Speed 

m 

■ 

0.5 

m 

■ 

0.5 

n 

0.5 

A5  Difficult  scenario 

Uniform  forest 
Base  Locatio.i  F6 
Variable  wind 

Initial  wind  315  @  4  m/s 

New  Wind  225  §  18  m/s 

Time  of  Shift  T  =  6 
Rate  of  Shift  5 

First  Ignition  location  F14  at  time  T  =  0 
Second  Ignition  location  06  at  time  T  =  5 
Age  of  fire  when  reported  =  0 
Cost  Factors 

Watching  1 . 00 

Moving  2.00 

Mobilizing,  Demobilizing,  Fighting  4.00 
Clock  Period  »  10  seconds  between  updates 
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Fire  Fighting  Unit  Number  | 

1 

2 

3 

4 

5 

6 

7 

8 

Starting 

Position 

E5 

E6 

E7 

F5 

F7 

G5 

G6 

G7 

m 

m 

m 

1.1 

m 

m 

■1 

n 

Put -out 
Speed 

0.25 

■ 

0.25 

0.25 

m 

m 

0.25 

■ 
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APPBMDIX  B 


NRITTEB  MOUSE  PRACTICE  IHSTRUCTIOMS  PROVIDED  TO  SUBJECTS 

The  purpose  of  the  mouse  practice  facility  is  simply  to 
allow  you  to  become  familiar  with  using  this  particular 
mouse,  so  that  when  you  are  engaged  in  the  NEWFIRE  sessions 
you  will  not  be  distracted  from  solving  the  NEWFIRE  problem 
by  the  purely  mechanical  actions  of  manipulating  the  mouse. 

A  sample  mouse  practice  screen  is  on  the  next  page.  The 
mouse  practice  screen  consists  of  two  windows,  a  large  green 
one  and  to  the  right  of  it,  a  smaller  white  one.  Both  are 
divided  into  a  grid.  At  intervals,  a  light-blue,  niunbered 
icon  will  appear  in  the  green  grid.  You  are  to  move  that 
icon  to  the  smaller  blue  grid.  The  icons  are  moved  in  the 
following  manner. 

—  Click  once  on  the  icon  to  be  moved.  The  cursor  will 
become  a  cross. 

—  Place  the  cursor  (cross)  on  the  next  available 
destination  cell  (in  the  blue  field). 

—  Click  once.  The  icon  will  appear  in  that  cell,  and 
the  cursor  will  again  be  an  arrow. 

The  first  icon  you  move  must  go  in  the  upper  left-hand 
most  cell  of  the  white  destination  grid.  Each  successive 
icon  must  be  placed  in  the  cell  immediately  to  the  right  of 
the  one  just  filled.  When  a  row  is  full,  the  next  icon  goes 
in  the  left-hand-most  cell  of  the  next  row.  The  order  of 
the  numbers  on  the  icons  is  unimportant.  That  is,  you  may 
fill  the  top  row  with  the  sequence  1,3,2,6,5,4,8,12,10. 

You  are  now  ready  to  begin  the  mouse  practice. 

o  Move  the  cursor  to  the  blue  field  on  the  right  hand 
side  of  the  screen. 

o  Click  the  right  mouse  button  once.  A  menu  will 
appear . 

0  Move  the  cursor  down  to  highlight  "mouse  practice". 

o  Click  once  with  the  left  mouse  button. 

The  mouse  practice  begins  immediately. 

After  you  have  completed  the  mouse  practice: 

o  Ensure  the  proctor  records  your  time  score. 

o  Click  once  with  the  left  mouse  button  in  the  small 
blue  square  (with  a  diagonal  yellow  line)  in  the  upper  left 
corner  of  the  screen. 
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APPENDIX  C 


NENPIRE  INSTRUCTIONS  PROVIDED  TO  SUBJECTS 

Cl  Written  instructions  provided  before  beginning  practice 
scenario (s) 

INTRODUCTION. 

The  purpose  of  this  experiment  is  to  examine  how  people 
make  decisions  in  complex  dynamic  environments.  We  will  use 
a  simulated  fire-fighting  environment  called  NEWFIRE.  No 
experience  or  knowledge  of  fire-fighting  is  needed. 

Below  is  a  (green)  area  occupying  the  left  and  middle 
part  of  the  computer  screen  from  bottom  to  top.  This  is  a 
map  of  a  forest.  The  map  is  divided  into  18  x  18  squares  or 
cellS;  each  representing  several  hectares  of  vegetation  (1 
hectare  =  10,000  square  meters  or  2.5  acres). 

The  forest  is  monitored  by  an  airplane.  If  the  pilot 
observes  fire  in  the  forest,  he/she  will  transmit  the 
coordinates  of  the  place  or  places  where  it  burns.  A  cell 
containing  ignited  forest  changes  color  from  green  to  red  on 
your  map. 


Figure  1.  Sample  NEWFIRE  Screen 
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Some  fires  are  confined  to  a  single  forest  cell  because 
they  were  discovered  immediately.  Others  extend  over  a 
cluster  of  cells  because  they  were  ignited  before  the 
spotter  plane  arrived  in  the 
area. 

In  any  of  your  sessions  it  is  vital  for  you  to  prevent 
the  fire  from  reaching  a  particular  cell  on  the  map.  This 
cell  is  brown  and  has  a  B  written  on  it.  B  stands  for  base 
and  marks  out  the  place  where  you  are  situated.  If  the  base 
is  lost,  the  session  will  terminate  immediately. 

You  are  to  perform  your  fire  fighting  task  by  acting  as 
a  commander  of  eight  fire  fighting  units.  These  are  squads 
of  forest  rangers  moving  in  Landrovers.  The  cells  in  which 
these  resources  are  positioned  are  overlaid  by  an  icon  which 
you  can  click  on  with  the  mouse.  The  icon  of  a  fire 
fighting  unit  shows  the  unit's  identification  number  and 
changes  color  in  accordance  with  the  unit's  activity.  Two 
or  more  units  may  be  in  the  same  cell  at  the  same  time.  If 
two  or  more  units  are  in  a  cell,  the  icon  displayed  belongs 
to  the  unit  that  entered  that  cell  last. 

In  the  upper  right  corner  of  the  screen  you  can  read  the 
direction  and  speed  (in  meters  per  second)  of  the  wind.  The 
arrow  of  the  wind  indicator  points  to  the  direction  the  wind 
is  blowing  toward.  The  numbers  on  the  circular  scale  of  the 
indicator  can  be  used  to  read  the  compass  direction  the  wind 
is  blowing  from.  See 
Figure  2.  (next  page) 

The  middle  panel  to 
the  right  of  the  forest 
map  is  the  Message  Panel. 

Messages  from  the  units 
are  displayed  here  as 
text.  This  panel  also 
lists  the  current 
location  of  each  unit. 

Fortunately,  the  fire 
fighting  units  are 
thoroughly  trained  on 
their  job.  They  perform 
it  according  to  a  fixed 
schedule  within  the  cells 
to  which  they  have  been 
commanded  to  go.  The 
only  thing  you  have  to  do 
is  to  deploy  and  redeploy 
the  units.  You  may  wish 
to  direct  a  particular 
unit  to  a  particular  cell 
either  because  fire  has 

been  ignited  within  the  cell,  or  because  the  forest  in  the 
cell  is  threatened  by  approaching  fire.  Whatever  your 
intention  is,  the  unit  will  take  proper  action  when  it 


released  in  the  center  of  the 
forest  map  would  travel  to  the 
lower  left  hand  corner  of  the 
screen. 


47 


arrives  at  the  destination.  If  the  unit  detects  fire  in  its 
new  position,  it  will  start  fighting  the  fire  inunediately. 
Otherwise  the  unit  will  send  the  message  'watching*  and  keep 
itself  ready  to  fight  any  fire  that  may  be  ignited  in  the 
cell  at  a  later  time.  When  you  begin  a  session,  all  fire 
fighting  units  are  inactive. 

OPERATING  INSTRUCTIONS. 

O  TO  MOVE  A  FFU: 

—  Place  cursor  on  FFU  to  be  moved. 

—  Click  once  with  left  mouse  button.  The  cursor  will 
now  become  a  cross. 

—  Place  this  cross  on  the  cell  you  want  the  FFU  to  go 
to. 

—  Click  once  with  the  left  mouse  button.  The  number  of 
unit  selected  will  appear  in  the  upper  left  hand 
corner  of  the  destination  cell. 

The  unit  will  change  color  to  purple  to  indicate  it  is 
travelling  to  its  destination.  Also,  the  message  "MOVING" 
will  appear  in  the  message  panel  to  the  right  of  the  forest 
map.  The  unit  will  not  however  appear  to  move  until  the 
next  update. 

0  TO  CHANGE  THE  DESTINATION  OF  A  TRAVELLING  FFU; 

—  Place  cursor  on  the  purple  icon  of  the  unit  itself, 
NOT  the  destination  cell. 

—  Click  once  with  left  mouse  button.  The  cursor  will 
now  become  a  cross. 

—  Place  the  cross  on  the  new  cell  you  want  the  FFU  to 
go  to. 

—  Click  once  with  the  left  mouse  button.  The  number  of 
unit  selected  will  appear  in  the  upper  left  hand 
corner  of  the  new  destination  cell. 

O  TO  STOP  A  TRAVELLING  UNIT,  AND  KEEP  IT  WHERE  IT  IS: 

—  Place  cursor  on  the  purple  icon  of  the  unit  itself. 

—  Double  Click  the  left  mouse  button. 

O  IF  TWO  OR  MORE  FFU's  ARE  IN  ONE  CELL; 

—  Move  cursor  to  the  selector  pane  in  the  lower  right 
hand  corner  of  the  screen. 

—  Click  once  on  the  line  containing  the  unit  you  wish 
to  move.  The  cursor  will  become  a  cross. 

—  Place  the  cursor  on  the  cell  you  want  that  FFU  to 
move  to.  The  FFU's  number  will  appear  in  the  upper 
left  hand  corner  of  the  destination  cell. 

If  you  try  to  click  on  the  icon  of  a  unit  that  is  not 
the  only  one  in  the  cell,  you  will  be  prompted  to  make  your 
choice  of  units  in  the  selector  pane. 
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Don't  worry  that  the  units  may  have  to  pass  through 
burning  forest  in  order  to  reach  their  destinations.  The 
cells  are  large  and  never  on  fire  everywhere  at  the  same 
time,  despite  their  uniform  red  color  on  the  map. A  unit 
begins  fighting  fire  by  mobilizing  its  resources:  The 
forest  rangers  pull  out  their  hoses  and  start  the  pump 
engine.  Then  the  unit  extinguishes  the  fire.  When  all  fire 
has  been  extinguished,  the  unit  demobilizes:  The  forest 
rangers  dismantle  their  equipment  and  return  to  the 
Landrovers.  The  unit's  icon  is  dark  blue  during  these 
operations,  which  can  be  monitored  by  reading  a  message  the 
unit  sends  to  you  each  time  it  initiates  a  new  action.  Your 
screen  is  updated  every  hour  on  the  hour.  You  cannot 
interrupt  the  fire  fighting,  nor  can  you  speed  it  up  by 
letting  several  units  extinguish  simultaneously  in  the  same 
cell.  If  you  place  a  unit  in  a  cell  in  which  another  unit 
is  fighting  fire,  the  former  unit  will  perceive  that 
deployment  as  a  parking  maneuver  and  not  begin  to  mobilize. 
You  are  allowed  to  provide  a  unit  engaged  in  fire  fighting 
with  a  new  destination.  The  unit  will  obey  the  command  as 
soon  as  i'.  has  extinguished  and  demobilized.  A  unit 
retained  in  its  position  after  a  completed  fire  fighting 
operation  becomes  inactive. 

Once  a  fire  fighting  unit  has  extinguished  a  fire  that 
cell  will  appear  as  a  gray  square  when  the  unit  is  moved  to 
a  new  positions.  Unattended  fires  burn  out  after  some  time 
and  leave  black  squares  on  the  map.  The  residue  of  the 
forest  in  these  black  or  gray  cells  is  not  flammable  and 
will  not  re-ignite. 

A  trial  terminates  automatically  when  there  is  no  more 
fire  in  the  forest,  or  when  you  lose  the  base,  whichever 
comes  first.  Only  the  portion  of  a  cell  that  burns  is 
counted  as  a  loss. 

The  scenarios  you  will  encounter  in  your  sessions  have 
been  selected  at  random.  The  base  and  fire  fighting  units 
may  be  positioned  anywhere  on  the  forest  map,  and  a  fire  may 
be  displayed  at  any  moment  in  any  cell  or  cluster  of  cells. 

In  some  scenarios  another  fire  will  be  ignited  after  a 
while,  but  it  also  is  possible  that  there  will  only  be  the 
initial  fire  to  cope  with.  The  potential  secondary  fire  is 
always  confined  to  a  single  forest  cell. 

The  direction  and  speed  of  the  wind  vary  from  one 
scenario  the  other.  The  wind  may  remain  constant  throughout 
a  trial,  or  it  may  shift  more  or  less  suddenly,  but  not  more 
than  once. 

O  TO  REMOVE  THE  SCREEN  DISPLAYED  AFTER  A  COMPLETED  TRIAL: 

—  Move  the  cursor  to  the  label  bar  uppermost  on  the 
screen. 

—  Click  with  the  right  mouse  button,  or  press  the  Del 


49 


key  to  open  a  menu  that  contains  a  line  labelled 
•close' . 

—  Position  the  cursor  on  this  line,  and  click  with  the 
left  mouse  button  to  execute  the  close  command. 

Good  Luck! 


You  are  now  ready  to  perform  a  short  practice  scenario 
to  familiarize  your  self  with  the  NEWFIRE  environment  and 
its  operation. 

o  Open  the  menu  by  a  right  mouse  click  in  the  blue 
region  of  the  screen. 

o  Move  the  cursor  down  to  highlight  "NEWFIRE". 

0  Click  on  "NEWFIRE"  once  with  the  left  mouse  button. 

0  Insert  the  diskette  provided  to  you  by  the  proctor. 

0  Click  once  with  the  left  mouse  button  on  the  bar 
prompting  you  for  the  diskette. 

0  Click  once  with  the  left  mouse  button  on  "Go  Ahead", 
o  Select  "Proceed  with  your  task  number  1"  by 

highlighting  that  bar  and  clicking  once  with  the 
left  mouse  button. 

The  simulation  begins  ismiediately . 

When  you  have  completed  the  practice  scenario,  a  yellow 
"GAME  TERMINATED"  banner  appears  across  the  top  of  the 
screen.  Return  to  the  start-up  screen  by  clicking  once  on 
the  small  blue  square  in  the  upper  left  hand  corner. 

Ask  the  proctor  for  further  instructions  before  continuing. 


C2  Instructions  given  to  subjects  after  completion  of 
practice  scenario(s) 

You  are  now  ready  to  begin  the  experiment  scenarios. 

o  Select  "Proceed  with  your  task  number  2"  by  clicking 
on  that  line  with  the  left  mouse  button. 

o  After  completing  that  scenario,  return  to  the  start¬ 
up  screen  as  before,  and  select  "Proceed  with  task  nximber 
3".  Then  perform  task  number  4,  etc. 

o  After  all  tasks  have  been  completed,  the  proctor  will 
give  you  a  short  questionnaire  to  fill  out.  That  will 
conclude  the  experiment. 

Thank  you  very  much.  Your  time  and  efforts  are  greatly 
appreciated . 
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APPEMOIX  D 


OBJECTIVE  AMD  COST  IMPORMATIOM  PROVIDED  TO  SUBJECTS 

D1  Objective  information  provided  to  the  one  objective 
group 


Your  mission  is  to  put  out  all  the  fires  without  losing 
your  base.  In  extinguishing  the  fire,  you  should  mlniaiize 
the  amount  of  forest  area  burned. 

Note  that  s<nBe  fire-fighting  units  may  not  be  as  effective 
as  others  with  regard  to  the  speed  with  which  they  move,  and 
the  speed  with  which  they  put  out  a  fire. 


D2  Objective  information  provided  to  the  two  objective 
group.  The  order  the  objectives  were  presented  was 
randomized  within  the  group. 

So  far  the  cost  of  operating  the  fire-fighting  units 
was  not  an  issue.  However,  in  the  real  world,  these  cost 
are  constraints  on  decision-makers.  This  simulation  takes 
that  into  consideration  by  accumulating  costs  at  the  rates 
listed  below. 


FFU  Activity 

Inactive  . 

Watching  . 

Moving  . 

Mobilizing,  Fightin j  Fire, 
or  Demobilizing  . 


Cost  per  Time  Period 
...  No  Cost 
. . .  $  1.00 
. . .  $  2.00 

. $  4.00 


Your  mission  is  to  put  out  all  the  fires  without  losing 
your  base.  In  doing  this,  you  should 

o  minimize  the  amount  of  forest  area  burned, 
o  minimize  costs  incurred  by  PFU's 


Both  of  these  objectives  are  of  equal  importance. 

Note  that  some  fire-fighting  units  may  not  be  as  effective 
as  others  with  regard  to  the  speed  with  which  they  move,  and 
the  speed  with  which  they  put  out  a  fire. 
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D3  Objective  and  cost  information  provided  to  the  three 
objective  group.  The  order  the  objectives  were  presented 
was  randomized  within  the  group. 

So  far  the  cost  of  operating  the  fire-fighting  units 
was  not  an  issue.  However,  in  the  real  world,  these  cost 
are  constraints  on  decision-makers.  This  simulation  takes 
that  into  consideration  by  accumulating  costs  at  the  rates 
listed  below. 


FFU  Activity  Cost  per  Time  Pe 

Inactive  .  No  Cost 

Watching  .  $  1.00 

Moving  .  $  2.00 

Mobilizing,  Fighting  Fire, 

or  Demobilizing . $  4.00 


Your  mission  is  to  put  out  all  the  fires  without  losing 
your  base.  In  doing  this,  you  should 

o  extinguish  the  fire  in  as  little  time  as  possible 
o  minimize  costs  incurred  by  PFU's 
o  minimize  the  amount  of  forest  area  burned. 

All  three  of  these  objectives  are  of  equal  importance. 

Mote  that  some  fire-fighting  units  may  not  be  as  effective 
as  others  with  regard  to  the  speed  with  which  they  move,  and 
the  speed  with  which  they  put  out  a  fire. 
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JLPPEVDIX  E 

QUBSTIOMS  AMSNBRED  BT  SUBJECTS 

Ai  Questions  answered  by  all  subjects. 

1.  Describe  the  general  strategy  you  followed  when 
deploying  Fire  Fighting  Units  (FFU's)? 


2.  Please  try  to  elcJsorate  on  the  thinking  process  you  went 
through  in  making  your  decisions  in  this  session.  (Use  the 
back  of  the  page  if  necessary) 


3.  Describe  how  the  goals  you  were  given  affected  your 
strategy,  or  how  you  incorporated  those  goals  in  your 
strategy . 
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7.  How  clear  were  the  instructions  regarding  the  task? 


1  2  3  4  5  6  7 

Mot  at  all 
Clear 


8  9 

Very 

Clear 


8.  To  what  extent  was  the  information  provided  in  the 
message  panel  (along  the  right  hand  side  of  the  screen) 
helpful  in  making  decisions? 

123456789 
Not  at  all  Very 

Helpful  Helpful 


9.  To  what  extent  was  the  report  of  time  (across  the  top  of 
the  screen)  helpful  in  making  decisions? 

123456789 
Not  at  all  Very 

Helpful  Helpful 


10.  To  what  extent  was  the  report  of  cost  (across  the  top  of 
the  screen)  helpful  in  making  decisions? 

123456789 
Not  at  all  Very 

Helpful  Helpful 


11.  To  what  extent  was  the  report  of  area  lost  (across  the 
top  of  the  screen)  helpful  in  making  decisions? 

123456789 
Not  at  all  Very 

Helpful  Helpful 


12.  In  the  sessions  you  just  completed,  did  you 

a.  Use  the  cost  information  (Y/N)? 

b.  If  you  did,  please  describe  how  you  used  that 
information . 
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13.  Briefly  describe  any  previous  experience  you  have  had 
with  fire-fighting. 


14.  How  interesting  was  the  task  you  just  performed? 

4  5  6  7  8 


12  3 

Not  at  all 
Interesting 


9 

Very 

Interesting 


15.  How  serious  were  you  in  performing  the  task? 

4  5  6  7  8 


12  3 

Not  at  all 
Serious 


9 

Very 

Serious 


16.  How  clear  were  the  instructions  regarding  the  task, 
generally? 


12  3 

Not  at  all 
Clear 


8 


9 

Very 

Clear 


17.  How  easy  was  the  system  to  use? 

12345678 
Not  at  all 
Easy 


9 

Very 

Easy 


18.  How  familiar  are  you  with  computers,  generally? 


12  3 

Not  at  all 
Feuniliar 


8 


9 

Very 

Feuniliar 


19.  How  familiar  are  you  with  using  direct  manipulation 
devices  (e.g.  mouse,  joystick)? 


12  3 

Not  at  all 
Familiar 


9 

Very 

Feuniliar 


20.  Please  give  me  some  information  about  yourself  (in 
absolute  confidence).  At  no  time  will  your  name  appear  in 
the  results.  The  data  will  only  be  used  in  an  aggregate 
statistical  sense. 

a.  Curriculxun: 

b.  Branch  of  Service: 

c .  Community 

d.  Sex: 

e .  Age : 

f.  Years  of  full  time  work  experience: 

g.  How  many  years  ago  did  you  complete  your 
undergraduate  education : 

21.  Your  general  comments  regarding  the  simulation: 
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A2  Additional  questions  answered  by  the  two  objective  group 


4.  In  executing  the  scenarios,  describe  how  you  weighted 
cost  relative  to  area: 

a.  Substantially  less  than  area 

b.  Somewhat  less  than  area 

c.  Equally 

d.  Somewhat  more  than  area 

e.  Substantially  more  than  area 

Any  comments: 


5.  For  each  scenario,  was  the  target  cost  given  adequate  to 
accomplish  the  task? 


a.  Scenario  #  1 


1  2  3  4  5  6  7 

Not  at  all 
Adequate 


8  9 

Very 

Adequate 


b.  Scenario  #  2 


1  2  3  4  5  6  7 

Not  at  all 
Adequate 


8  9 

Very 

Adequate 


c.  Scenario  #  3 


1  2  3  4  5  6  7 

Not  at  all 
Adequate 


8  9 

Very 

Adequate 


6.  For  each  scenario,  was  the  upper  limit  (ceiling  cost) 
given  adequate  to  accomplish  the  task? 

a.  Scenario  #  1 


1  2  3  4  5 

Not  at  all 
Adequate 


6 
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7 


8 


9 

Very 

Adequate 


b.  Scenario  #  2 


123456789 
Not  at  all  Very 

Adequate  Adequate 

c.  Scenario  #  3 

123456789 
Not  at  all  Very 

Adequate  Adequate 


A3  Additional  questions  answered  by  the  three  objective 
group 

4a.  In  executing  the  scenarios,  describe  how  you  weighted 
cost  relative  to  area: 

a.  Substantially  less  than  area 

b.  Somewhat  less  than  area 

c.  Equally 

d.  Somewhat  more  than  area 

e.  Substantially  more  than  area 


4b.  In  executing  the  scenarios,  describe  how  you  weighted 
cost  relative  to  time: 

a.  Substantially  less  than  time 

b.  Somewhat  less  than  time 

c.  Equally 

d.  Somewhat  more  than  time 

e.  Substantially  more  than  time 


4c.  In  executing  the  scenarios,  describe  how  you  weighted 
area  relative  to  time: 

a.  Substantially  less  than  time 

b.  Somewhat  less  than  time 

c.  Equally 

d.  Somewhat  more  than  time 

e.  Substantially  more  than  time 

Any  comments: 
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5*  For  each  scenario,  was  the  target  cost  given  adequate  to 
accomplish  the  task? 

a.  Scenario  #  l 

123456789 
Not  at  all  Very 

Adequate  Adequate 


b.  Scenario  §  2 


12345678 
Not  at  all 
Adequate 


9 

Very 

Adequate 


c.  Scenario  #  3 


12345678 
Not  at  all 
Adequate 


9 

Very 

Adequate 


6.  For  each  scenario,  was  the  upper  limit  (ceiling  cost) 
given  adequate  to  accomplish  the  task? 

a.  Scenario  #  i 


123456789 
Not  at  all  Very 

Adequate  Adequate 

b.  Scenario  #  2 

123456789 
Not  at  all  Very 

Adequate  Adequate 


c.  Scenario  #  3 

123456789 
Not  at  all  Very 

Adequate  Adequate 
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